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A method of image registration and medical image data processing apparatus 
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The present invention relates to image registration, and more particularly 
without limitation, to image registration in medical diagnostic systems. 

-In-Gurrent-cliniGal-practice,-it is not uncommon that several-medicaLimages-are 

* 

to be compared or combined* For example, this occurs when images of a single individual are 
5 acquired by different techniques, each producing different information about the^individual^ 
One technique may depict brain function very well and another a brain tumour. 

The combination of the two is crucial in determining whether the tumour can 
be resected without damaging vital brain areas. A second example is that of images of an 
individual taken over a period of time. This can be used, among other things, to study the 
1 0 development of anatomical structures. Besides the comparison of images of a single 

individual, comparisons of images of different individuals is also performed, e.g. to study the 
differences and similarities between groups. 

To be able to compare images, the contents of the images must be in 
alignment This is often not the case for numerous reasons, such as movement of the 

1 5 individual or differences in the field of view (the part of the anatomy that is captured in the 

<- 

images). The process of finding the correspondence between the contents of images is called 
image registration. This is achieved by optimizing a measure ofsimHarity between images, 
such as the position of corresponding landmarks or the similarity in intensity of 

« 

corresponding anatomical structures. 
20 In most cases, global translations and rotations suffice to align images. When 

. one image has to be deformed, however, to fit the contents of the other ; this is referred to as 

non-rigid or elastic registration. This is necessary for registration of images of different . 

i . ( 

individuals (who have different anatomies) or images of a single individual when an 
anatomical change has taken place. • • •; 

25 

Various image registration techniques are known from the prior art, such as 
from WO 01/ 01346 Al, EP 1 184 802 A2, US 2002 / 0 141 626 Al., US 6,266,453 Bl. 
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The book Medical Image Registration" by Josef Hajnal provid es an overview 

0t me Various P rior 3X1 registration techniques. As far as elastic image registration is 

concerned the article ''Elastic Image Registration and Pathology Detection" by Paul 
Thompson and Arthur W. Toga 

* 

5 (htl p://ww.loni. U c la.ed u/^ho m p S o n ^>DP/no jics_Elas Chpt.pdf) provides an overview of 
ffieknown approaches fore TasUo image registration. 1ms article is also published as a book 
chapter in "Handbook of Medical Image Processing", Academic Press, 2000. 

A-cc^mwm^vantage-of prior art image registration metodsTsliie-lack-of— 
an objective quality criterion. So far, the only reliable way to check the results of the image 

registration is visual assessment by ahuman observer. This is a practical problem that _. 

substantially limits the appUcation of registration methods. For example, in case of time 
series where many sequential images have to be registered it is impractical to interrupt an 
otherwise automatic pre-processing method. This is also true if registration is part of a pre- 
processing chain. 



15 



20 



25 



30 



It is therefore an object of me present invention to provide for an improved 
method of image registration which addresses the problems of the prior art. It is a further 
object of the present invention to provide for an improved computer program product and 
improved medical image data processing apparatus for image registration. 

The present invention provides for a n improved method of image registration 



where registrations are performed for a number of sub-volumes of the image to be registered 
in addition to a global registration. For example, a number of sub-volumes contained in the 
image data is selected manually by an operator by marking the sub-volumes of interest. 

i^ or m ^ s P^P 086 a graphical user interface (GUT) can be utilized. . Alternatively 
or in addition an automatic image segmentation can be performed for identification of such 

sub-volumes, For example, each of the sub-volumes contains a certain medical region of 

interest, such as an organ or a tumour. ,,,, ^ v . .. " ". 

In accordance with the present invention a registration is performed for each - 
one of the .sub-volumes in addition to a global registration of the image data. This enables to 
identify an outlier sub-volume which substantially deviates from the transformation of the 
other sub-volumes and the global registration. Such an outlier sub-volume can be cut off in 
order to improve the overall registration result or it can trigger a subsequent elastic ■ , 



-•> i. 



* ■, . . 4 ^ - . W 
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registration for improved registration. The transformation parameter set obtained in the first 
registration step can be used as an input parameter set for the subsequent elastic registration. 

The present invention is particularly advantageous as it registers sub-volumes 
of the images and translates the registration results to global transformations. In particular 
this enables a comparison of the resulting transformations which indicates: 

the accuracy of the global transformation, e.g. by measuring variations, 

the reason for the inaccuracy, e.g. too much constraint transformation, 
~ the location of the problem-in-th^mTa^r^grlocal image artefacts. 

In some applications outliers in the registration of the sub-volumes are likely 
to originate from image artefacts or unequal anatomies in the-images such as pathology, 
especially if the structures can be assumed as rigid, such as the scull. An example is neural- 
MR where a sub-volume is selected on a tumour that does not exist on the other image. 
Another example is neural-CT where a sub-volume is selected on image artefacts caused by 
metal fillings. A possible reaction to this detected origin of inaccuracy can be to signal for 
user-interaction or to cut-off this section for a more accurate global registration. 

If the image is to be registered e.g. in the abdomen with the deforrning organs 
such as the liver an additional processing step is performed for improved registration rather 
than cutting-off the corresponding sub-volume. An inaccurately registered sub-volume can be 
caused by deformation of the organ, e.g. due to breathing motion, which cannot be 
appropriately dealt with by a rigid, global transformation. In this case a refinement and / or 
localization of the transforma t ion is p erformed in order to improve the registration of this 
sub-volume. For this improved registration the transformation can be locally extend7d"with 
more degrees of freedom; in particular non-rigid or elastic image registration can be used for 
this purpose. 

Another advantageous application of the present invention is the detection of 
artefacts in MR inter-leaved scans, i.e..the even and odd slices being not aligned. Image 
interpolation between the slices results in double edges and other artefacts which make 
registration inaccurate and can result in a low quality after the reformatting which is 
subsequent to the registration. The present invention enables to detect such artefacts in orders 
to warn a user that artefacts are present in the MR inter-leaved scans. 

In essence, the invention provides for a sub-regional registration to be 
performed alongside the global registration of the full images. Sub-regions or sub-volumes, 
of the images to be registered are selected and are registered throughout the sequence of 
images. In accordance with a preferred embodiment a table of registration information is 
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compiled for all sub-regions and the global registration containing translation and rotation 



mformation. Quantification of toe registration process is therefore possib le over sub-regions 
and allows identification of inaccuracies within the overall registration process. 

In particular, this enables to draw the users attention to the sub-volume or sub- 
5 volumes of the global image where the registration process has failed and manual orientation 
isTe^uu^dTTatem^^ 



incorporated into a fully automated image handling sequence which enables spurious sections 
ofth^eqnenc^TDfitnages to be removed. — 

10 

^^^^^^ * ^- • *■ « 

In the following preferred embodiments of the invention are explained in 
greater detail by making reference to the drawings in which: 

Fig. 1 is illustrative of a flow diagram of an embodiment of a method of the 

invention, 

1 5 Fi & 2 is a schematic example for the registration of sub-volumes, 

Fig. 3 is a block diagram of a medical image data acquisition and processing 

system 



20 Fi & 1 shows a flow diagram of an embodiment of the registration method of 

the invention. In step 100 im age data is provided by an imaging modality. Depending on the 

application all of the image data are provided by the same image modality. Alternatively the 

image data of different image modalities are to be combined. For example, X-ray imaging, 

magnetic resonance imaging (MRI), computer tomography (CT) imaging, functional MRI 

25 (fMRT), single photon emission computer tomography (SPFCT) or positron eniission 

tomography (PET) are used as image modalities. . . .. i. , .,. . , 

■ ' ■ ■ 

In step 1 02 a selection of subsumes Si m the image data is performed; This 

can be done manually by marking the sub-volumes Si on a display or by performing an image 
segmentation step or another suitable image processing step for identification of subrregiohs 
3 0 or sub-volumes in the image data: >L 

■ • 

In step 1 04 image registrations are performed locally for the selected sub- 
volumes Si. For example, these registrations are performed by means of a rigid image 
registration method. In step 106 a parameter set Tj is oufputted for each one of the sub- 

» 

volumes S { for which a registration has been performed. Each one of the transformation 
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parameter sets Ti is descriptive of the displacement of the corresponding sub-volume S\ in the 
two images to be registered. 

In step 108 a global registration is performed for the images to be registered. 
Again a rigid image registration method can be employed for this purpose. In step 110 a 
5 global transformation parameter set Tq is outputted. The global transformation parameter set 
To is descriptive of the displacement of the images to be registered with respect to each other. 

In step 1 12 the index i is initialized to be equal to one. 
Ea~st^"lT4"arrnean distance measure is calciilated~fbr r thriransfCTffljaLliTar — 
parameter set Ti. This can be done by calculating a distance measure between the 

1 0 transformation-parameter- set Ti and each one of the other transformation parameter- sets Tj 

and to the global transformation parameter set Tq and calculating a mean value on the basis 
of these individual distance measures. For this purpose any distance measure for the 
determination of the distance between two parameter sets or vectors can be used, such as 
Euclid's distance. For example Euclid's distance is calculated for all pairs of parameter sets 
15 Ti , Tj, where i * jr. and the resulting Euclid's distances are added. This provides the mean 

distance measure. 

■ 

In step 1 1 6 it is determined whether the mean distance measure of step 1 14 is 
above a threshold level. If this is not the case the registration of the corresponding sub- 
volume Si is deemed to be sufficiently accurate and the control goes to step 120 to increment 
20 the index i. 

» • V 

If the mean distance measure is above the threshold the control goes from step 
1 16 to step 1 18. In step 1 18 an output signal is generated for identification of the sub-volume 
Si. This output signal can be outputted as a warning signal by means of a graphical user 
interface in order to warn an operator that the registration of the sub-volume Si is inaccurate. 
Alternatively or in addition the output signal generated in step 1 1 8 triggers the execution of 

■ 

an additional non-rigid, elastic image registration procedure; for improved registration of the - 
sub-volume Si and / or for improved global registration. ; ^ vh > 

As an alternative to steps 1 14 and 116 another statistical method can be used 
to identify an outlier parameter set and corresponding sub-volume. 

Fig. 2 shows a schematic example. The example is illustrative of an 
application in two dimensions for convenience of explanation. It is to be noted that the 
example can be extended to more dimensions. 
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Image 100 is a slice or "slab" of image data which has been acquired by 



medicanmage data acquisition. Image 100 is located i n the XY-plane 104. Image 102 i s also 
within plane 104 and needs to be registered with image 100. 

hnage 100 contains the sub-volumes A, B, C and D. These sub-volumes are 
5 selectedby marking me corresponding regions in image 100 by means of a graphical user 
interface by an operator or by a segmentation step or another suitable data processing 



A', B\ C and D»r 



;sponding 



An image registration is performed both globally for images 100 and 102 and 
for the individual sub-volumes. This means that me sub-volumes A and Aiare registered^ 
well as the sub-volumes B-B', C-C and D-D'. 

Table 1 06 of Fig. 2 shows the resulting transformation parameter sets. In the 
example considered here the transformation parameter set for each individual image 
registration contains the amount of displacement in the X and Y directions within plane 104 
as well as the amount of angular transformation in degrees of rotation. 

From table 106 it is apparent that the transformation parameter set for the 
registration of the sub-volume B-B' substantially deviates from the other transformation 
parameter sets. The sub-volume B-B' is therefore identified as an outlier. In response a 
corresponding warning signal is outputted and / or an elastic registration procedure is 
triggered for improved registration of the sub-volume B-B'. The feet that the transformation 
parameter s et of the sub-volume B-B' is an outlie r can be determined by processing table 106 



25 



30 



by means of suitable statistical methods and / or by calculating o f distance measures. 

Fig. 3 shows a block diagram of a computer 200 which is coupled to a medical 
image data acquisition apparatus 202. Computer 200 has memory 204 for the storage of 
image data and has a graphical user interface 206. Further computer 200 has segmentation 
program module 208> image registration program module 210, elastic ima^ re^stration 
program module ?12 and memory 214 for the storage of fransformatibnyar^meters (cf. table 
106 of Fig; 2). hi addition computer 200 has program 216 for corifrollihg: the operation of 
these various components. Computer 200 is coupled to display 218 for rendering of image . , 
data. Alternatively or in addition computer 200 is connected to a printer.-,,. 

In operation the image data is loaded into memory 204 from image data 
acquisition apparatus 202. The image data is rendered on display 218, By means of graphical 
user interface 206 ah Operator can select a number of sub-volumes in the image data. 
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Alternatively or in addition sub-volumes are selected automatically by means of 
segmentation program module 208. 

Image registrations are performed by means of program module 210 for 
registering of the sub-volumes and for performing a global image registration. The 
corresponding transformation parameters are stored in memory 214 for analysis by program 
216. When program 216 identifies an outlier sub-volume a warning message is generated and 
outputted on display 218. Alternatively or in addition program 216 triggers program module 
212 for perfonning an elastrcimag^ sub-volume. 
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LIST OF REFERENCE NUMBERS: 



100 image 

102 image 

104 plane 

106 table 



5 200 computer 

202 image data acquisition apparatus 

204 memory 

206 GUI 

208 segmentation program module 

10 210 program module 

212 program module 

214 memory 

216 program 

218 display 
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1* A method of image registration comprising the steps of: 

providing at least first image data and second image data, 
selecting of sub-volumes of the first and the second image data. 



performing a registration for each one of the sub-volumes, each registration 
5 providing a transformation parameter set, 

™ ^performing a global registration for the first and second image data, the global 
registration providing a global transformation parameter set, 

comparing one of the transformation parameter sets to other transformation 
parameter sets and/or to the global transformation parameter set for identification of an 

■ 

10 outlier transformation parameter set of the transformation parameter sets, 

outputting of a signal being indicative of the sub-volume of the outlier 
transformation parameter set. 



2. The method of claim 1 , whereby the first image data and / or the second image 

1 5 data is provided by X-ray imaging, magnetic resonance imaging, computer tomography 

imaging, functional MRI, single photon emission computer tomography or positron emission 
tomo gr aphy. 



3. The method of claim 1 or 2, whereby the selection of the sub-volumes is 

20 performed manually by means of a graphical user interface. 



4. The method of claim 1, 2 or 3, whereby the selection of the sub-volumes is 

performed by means of an image segmentation step. 



f. • 



25 5, The method of anyone of the preceding claims 1 to 4, further comprising 

calculating a mean distance measure of the one of the transformation parameter sets to the 
other transformation parameter sets and to the global transformation parameter set, whereby 
the comparison of the one of the transformation parameter sets with the other transformation 
parameter sets and the global transformation parameter set is performed on the basis of the 
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mean distance me asure for detenruning whether the one of the transformation parameter sets 
is an outlier transformation parameter set. " 



6. The method of claim 5, whereby the one of the transformation parameter sets 
is identified as an outlier transformation parameter set, if the mean distance measure of the 
W5T ffie-traWfonrMion parameter-sefTiTpeMerlh^^ vaJue . 



7. 



"TheTnethod-of-anyone of the preceding claims 1 to-5 r further-eomprising- 



cutting off of the sub-volume of the outlier transformation parameter set. 



8. The method of anyone of the preceding claims 1 to 7, further comprising 
performing an elastic image registration in response to the signal. 

9. Computer program product, in particular digital storage medium, comprising 
program means for registration of at least first image data and second data, the program 
means being adapted to perform the steps of: 

storing of a selection of sub-volumes of the first and second image data, 
performing a registration for each one of the sub-volumes, each registration 
providing a transformation parameter set, 

performing a global registration for the first and second image data, the global 
registration providing a global transformation parameter set, 



comparing one of the transformation parameter sets to other transformation 
parameter sets and/or to the global transformation parameter set for identification of an 
outlier transformation parameter set of the transformation parameter sets, 

» ■ 

- , outputting of a signal being indicative of the sub-volume of the outlier 

transformation parameter set - w , 

r- , ' 

" " ' ■ ■ . •• 

10. Amedical image data processing apparatus, comprising: . .-....».<,.. 

a memory for a storage of at least first and second image data 

i • ■ • 

means for selecting of sub-volumes of the first and second image data, 
means for performing a registration for each one of the sub-volumes and for a 

global registration of the first and second image data, each registration providing a 

transformation parameter set, 
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means for identification of an outlier transformation parameter set of the 

transformation parameter sets on the basis of the transformation parameter sets, 

means for outputting of a signal being indicative of the sub-volume of an 

identified outlier transformation parameter set. 



» * 



1 1 . The medical image data processing apparatus of claim 10, further comprising 

a graphical user interface for selection of the sub-volumes. 



1 2 - medical image data processing apparatus of claims 1 0 or 1 1 , further 

1 0 comprising means.forimage segmentation, for the selection of the.sub-yolumes. _ 

* 

1 3 . The medical image data processing apparatus of claims 1 0, 1 1 or 1 2, further 

comprising means for elastic image registration for performing of an elastic image 
registration when an outlier transformation parameter set has been identified. 



< 
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ABSTRACT: EPO ~ 1 

13. 0UQ03 




The present invention relates to a method of image registration comprising the 

steps of: 

providing at least first image data and second image data, 



selecting of sub-volumes of the first and the second image data, 
5 - performing a registration for each one of the sub-volumes, each registration 

providing a transformation parameter sef," * 

performing a global registration for the first and second image data, the global 
registration providing a global transformation parameter set, 

comparing one of the transformation parameter sets to other transformation 
10 parameter sets and/or to the global transformation parameter set for identification of an 
outlier transformation parameter set of the transformation parameter sets, 

outputting of a signal being indicative of the sub-volume of the outlier 
transformation parameter set. 
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Global transformation parameter set T g 
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Determination of mean distance measure of transformation para- 
meter set T j to other transformation parameter set T j , where j * i, 

and to the global transformation parameter set Tq 
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